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SMALL  BUSINESS 

Action  Needed  to  Determine  Whether  DOD’s 
Comprehensive  Subcontracting  Plan  Test  Program 
Should  Be  Made  Permanent 


Why  GAO  Did  This  Study 

Since  1990,  DOD  has  been  conducting 
a  congressionally  directed  test 
program  related  to  how  contractors 
report  their  subcontracting  activities. 
The  purpose  of  the  program  is  to  test 
whether  using  comprehensive 
subcontracting  plans  that  cover 
multiple  contracts  across  contractor 
plants,  divisions,  or  entire  companies 
can  yield  administrative  cost  savings 
and  enhance  small  business 
subcontracting  opportunities.  Despite 
the  25-year  existence  of  the  program, 
little  is  publicly  known  about  its 
effectiveness. 

The  National  Defense  Authorization 
Act  for  Fiscal  Year  2015  included  a 
provision  for  GAO  to  report  on  the 
results  of  the  program.  This  report 
addresses  the  extent  to  which  the 
program  (1)  reduces  administrative 
costs,  and  (2)  enhances 
subcontracting  opportunities  for  small 
businesses.  GAO  analyzed  prior  DOD 
reviews  and  data  on  estimates  of 
administrative  costs  savings;  reviewed 
program  participants’  performance  for 
enhancing  small  business 
subcontracting  opportunities  for  fiscal 
years  2006  through  2013;  and 
interviewed  officials  from  DOD, 
program  participants,  and  small 
business  advocacy  groups. 

What  GAO  Recommends 

Congress  should  consider  making  the 
program  permanent.  GAO  also 
recommends  that  DOD  work  with 
Congress  on  the  program’s  status. 

DOD  agreed.  DOD  disagreed  with  a 
recommendation  to  draft  a  legislative 
proposal  to  make  the  program 
permanent.  GAO  subsequently 
modified  the  recommendation  and 
added  the  matter  for  Congress. 

View  GAO-16-27.  For  more  information, 
contact  William  T.  Woods  at  (202)  512-4841  or 

woodsw@gao.gov 


What  GAO  Found 

Reviews  commissioned  by  the  Department  of  Defense  (DOD)  concluded  that  the 
Test  Program  for  Negotiation  of  Comprehensive  Small  Business  Subcontracting 
Plans  (Test  Program)  has  resulted  in  the  avoidance  of  millions  of  dollars  in 
administrative  costs  for  both  program  participants  and  DOD.  According  to  the 
review  conducted  in  2013,  the  12  firms  then  participating  in  the  program  avoided 
about  $18.5  million  in  costs  through  the  use  of  single  comprehensive 
subcontracting  plans  rather  than  multiple  individual  subcontracting  plans.  Also,  a 
2007  review  estimated  that  DOD  avoided  administrative  costs  of  at  least  $45 
million  in  fiscal  year  2005.  GAO  reviewed  the  methodologies  used  for  these 
reviews  and  took  other  steps  to  validate  their  findings.  According  to  DOD 
officials,  if  the  Test  Program  were  terminated  or  allowed  to  expire,  a  significant 
one-time  administrative  cost  of  about  $22  million  could  result  to  participants. 
GAO’s  analysis  confirms  this  conclusion.  Test  Program  participants  and  DOD 
officials  GAO  interviewed  stated  that  the  program  also  has  resulted  in  non- 
financial  benefits,  including  greater  company-wide  awareness  of  small  business 
subcontracting  opportunities.  The  participants  GAO  interviewed  said  that  without 
the  program  their  companies  might  be  less  inclined  to  award  subcontracts  to 
small  businesses.  They  emphasized,  however,  that  the  program’s  continuing  test 
status  creates  uncertainty  and  inhibits  further  expansion.  The  2007  review 
recommended  that  DOD  work  with  Congress  to  make  the  Test  Program 
permanent;  however,  DOD  has  not  acted  on  this  recommendation.  Doing  so 
could  help  eliminate  uncertainty  with  the  program. 

GAO  found  that  the  Test  Program  enhanced  small  business  subcontracting 
opportunities,  although  participants’  performance  in  meeting  individual  goals  has 
varied.  Participants  are  evaluated  on  their  achievement  of  negotiated  initiatives 
and  goals  in  their  comprehensive  subcontracting  plans.  GAO’s  analysis  of 
performance  reports  found  that  participants  made  acceptable  progress  on  their 
initiatives  87  percent  of  the  time,  thus  providing  tangible  subcontracting 
opportunities  for  small  businesses.  For  example,  during  fiscal  years  2006  through 
2013,  program  participants  redirected  nearly  $93  million  in  subcontracts  from 
large  businesses  to  small  businesses.  Participants  also  achieved  a  72-percent 
success  rate  in  increasing  small  business  subcontracts  in  areas  such  as 
integrated  circuits  and  information  technology,  thus  addressing  a  concern  among 
some  small  businesses  that  high-end  technical  work  was  not  being 
subcontracted  under  the  program.  The  2013  DOD  review  estimated  that 
participant  initiatives  could  amount  to  as  much  as  $1 .8  billion  per  year  in 
increased  small  business  opportunities.  GAO’s  analysis  found  that  participants 
did  not  always  meet  individual  goals,  in  part  due  to  the  challenging  nature  of 
these  goals.  Flowever,  GAO  also  found  that  their  combined  performance  from 
fiscal  years  2006  through  2013  resulted  in  subcontract  awards  to  small 
businesses  that  exceeded  aggregate  goals  by  about  $5.4  billion.  The  annual 
performance  reviews  of  the  participants,  which  take  into  account  performance  on 
both  initiatives  and  goals,  have  been  largely  positive. 
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November  16,  2015 
Congressional  Committees 

Contractors  on  certain  defense  contracts  are  required  to  develop  plans 
specifying  how  much  of  the  work  will  be  performed  by  subcontractors.  In 
1989,  Congress  authorized  the  Test  Program  for  Negotiation  of 
Comprehensive  Small  Business  Subcontracting  Plans  (Test  Program), 
which  permits  participating  contractors  to  submit  a  single  subcontracting 
plan  covering  all  of  their  contracts  rather  than  a  separate  plan  for  each 
contract.  The  purpose  of  the  Test  Program  is  to  reduce  the  administrative 
burden  on  contractors  and  enhance  subcontracting  opportunities  for  small 
businesses.1  Although  the  Test  Program’s  initial  performance  period  was 
3  years,  Congress  has  extended  its  duration  eight  times,  with  the  most 
recent  extension  scheduled  to  end  December  31 , 2017.  Despite  the  25- 
year  tenure  of  the  Test  Program,  little  is  publicly  known  about  its  results, 
as  the  Department  of  Defense  (DOD)  has  not  published  any  reports  on  its 
performance. 

The  National  Defense  Authorization  Act  for  Fiscal  Year  2015  included  a 
provision  for  us  to  report  on  the  results  of  the  Test  Program.2  This  report 
addresses  the  extent  to  which  the  Test  Program  (1)  reduces 
administrative  costs,  and  (2)  enhances  subcontracting  opportunities  for 
small  businesses. 

To  address  whether  the  Test  Program  reduces  administrative  costs,  we 
analyzed  DOD-commissioned  reviews  of  the  program  performed  in  2002, 
2007,  and  2013;  each  of  which  included  estimates  of  the  cost  savings  or 
avoidance  associated  with  the  program.  We  assessed  the  methodologies 
employed  and  assumptions  used,  including  estimates  of  the  number  of 
contracts  and  cost  of  labor  involved  in  preparing  program  documentation, 
to  arrive  at  the  estimates  for  savings.  To  validate  the  data  used  in  the 


^he  National  Defense  Authorization  Act  for  Fiscal  Years  1990  and  1991,  Pub.  L.  No.  109- 
189,  §  834  and  the  National  Defense  Authorization  for  Fiscal  Year  1996,  Pub.  L.  No. 104- 
106,  §  811(a). 

2The  Carl  Levin  and  Howard  P.  “Buck”  McKeon  National  Defense  Authorization  Act  for 
Fiscal  Year  2015,  Pub.  L.  No.  1 1 3-291 ,  §  821  (e). 
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2013  study,  we  interviewed  DOD  officials  to  discuss  how  the  data  were 
collected  and  obtained,  and  we  analyzed  and  compared  the  data  to  more 
current  information  we  obtained  independently.  We  found  the 
methodologies  in  all  three  studies  to  be  reasonable,  and  the  data  were 
sufficiently  reliable  for  our  purposes.  We  interviewed  officials  from  DOD 
and  5  of  the  12  Test  Program  participants,  selected  to  represent  different 
levels  of  corporate  participation  in  the  program  and  varying  volumes  of 
small  business  subcontracting  activity,  to  obtain  their  views  on  any 
reduction  in  administrative  costs  and  other  benefits  they  may  have 
realized  from  their  participation  in  the  program.  The  views  expressed  by 
these  participants  provided  insight  into  the  operation  of  the  program,  but 
are  not  generalizable  to  all  program  participants. 

To  address  whether  the  Test  Program  enhances  subcontracting 
opportunities  for  small  businesses,  we  reviewed  legislation  and  agency 
guidance  for  the  Test  Program  and  relevant  GAO  and  DOD  reports.  We 
also  reviewed  Test  Program  participants’  comprehensive  subcontracting 
plans  from  fiscal  years  2006 — the  first  year  not  covered  by  a  prior  DOD 
review — through  2013 — the  last  year  for  which  full  data  were  available. 
We  compared  the  approved  subcontracting  goals  and  initiatives  as  stated 
in  the  plans  to  the  results  for  each  participant  and  as  reflected  in  annual 
performance  reviews,  known  as  Form  640  reviews,  conducted  by  the 
Defense  Contract  Management  Agency  (DCMA)  to  establish  trends  for 
participant  performance.  We  also  interviewed  officials  from  DOD’s  Office 
of  Small  Business  Programs,  DCMA’s  Small  Business  Programs  Division, 
the  five  Test  Program  participants  described  above,  and  two  small 
business  advocacy  groups  chosen  for  their  representation  of  the  small 
business  community  for  their  views  on  the  Test  Program.  The  views 
expressed  by  these  groups  provided  insight  into  the  operation  of  the 
program,  but  are  not  generalizable  to  all  small  businesses.  Appendix  I 
provides  additional  details  on  our  objectives,  scope,  and  methodology. 

We  conducted  this  performance  audit  from  March  to  November  2015  in 
accordance  with  generally  accepted  government  auditing  standards. 
Those  standards  require  that  we  plan  and  perform  the  audit  to  obtain 
sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for  our 
findings  and  conclusions  based  on  our  audit  objectives.  We  believe  that 
the  evidence  obtained  provides  a  reasonable  basis  for  our  findings  and 
conclusions  based  on  our  audit  objectives. 


Background 


In  fiscal  year  2014,  DOD’s  prime  contractors  subcontracted  for  $133 
billion  in  goods  and  services  to  support  DOD’s  missions,  of  which  $44 
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billion,  or  33  percent,  was  awarded  to  small  businesses.  The  Federal 
Acquisition  Regulation  (FAR)  generally  requires  proposed  prime 
contractors  to  have  individual  subcontracting  plans  in  place  for  contracts 
(including  modifications)  of  more  than  $650,000 — or  $1.5  million  for 
construction  contracts — whenever  subcontracting  opportunities  exist.3 
These  plans  are  to  document  subcontracting  goals  as  a  specific  dollar 
amount  planned  for  small  business  awards  and  as  a  percentage  of  total 
available  subcontracting  dollars  to  various  socioeconomic  categories  of 
small  businesses.4  The  plans  also  are  to  identify  the  types  of  products 
and  services  suitable  for  subcontracting  awards. 

Under  the  authorizing  statute,  as  implemented  in  regulation,  each 
participant  in  the  Test  Program  negotiates  and  reports  on  subcontracting 
goals  and  achievements  for  a  specific  fiscal  year  on  a  plant,  division,  or 
corporate-wide  basis.  A  comprehensive  plan  may  cover  a  large  number  of 
individual  contracts.  For  example,  one  participant  told  us  about  a  plan  that 
covered  more  than  3,000  contracts  that  otherwise  would  require  individual 
subcontracting  plans.  Reporting  small  business  subcontracting  activity  in 
a  comprehensive  plan  means  that  less  data  may  be  available  on  the 
subcontracting  activities  for  specific  contracts  or  programs.  In  addition, 
comprehensive  subcontracting  plans  are  to  include  various  initiatives  to 
enhance  small  business  subcontracting  opportunities  through  specific 
programs  or  other  actions.  According  to  a  DOD  official,  these  initiatives 
are  not  specific  to  any  one  contract,  but  can  be  completed  across  the 
entire  scope  of  defense  work  the  prime  contractor  performs.  Table  1 
highlights  the  key  differences  between  the  comprehensive  subcontracting 
plans  used  in  the  Test  Program  and  individual  subcontracting  plans. 


3FAR  19.707(a).  Effective  October  1, 2015,  the  prime  contractor  subcontracting  plan  floor 
will  be  raised  from  $650,000  to  $700,000  and  the  construction  threshold  of  $1,500,000  will 
stay  the  same.  80  Federal  Register  38293,  38294,  Final  Rule,  July  2,  2015. 

415  U.S.C.  §  644(g).  The  Small  Business  Act  defines  various  socioeconomic  categories  of 
small  businesses  including  women-owned,  veteran-owned,  service-disabled  veteran- 
owned,  small  disadvantaged,  and  Historically  Underutilized  Business  Zone. 
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Table  1:  Key  Differences  between  Comprehensive  Subcontracting  Plans  and  Individual  Subcontracting  Plans 

Comprehensive  subcontracting  plan 

Individual  subcontracting  plan 

Negotiation  of  plan  goals 

Each  fiscal  year 

Generally  once  for  the  duration  of  the  contract 

Time  frame  covered 

One  fiscal  year 

Duration  of  the  contract 

Plan  submission 

Once  per  fiscal  year 

Generally  once  for  the  duration  of  the  contract 

Inclusion  of  initiatives 

Yes 

No 

Defense  Contract  Management 
Agency  performance  evaluation 

Each  fiscal  year 

Every  third  fiscal  year  for  the  duration  of  the  contract 

Sources:  DOD  and  Federal  Acquisition  Regulation. 

|  GAO-16-27 

Current  Test  Program  eligibility  is  limited  by  statute  to  defense  contractors 
that  performed  at  least  three  DOD  prime  contracts  for  supplies  and 
services  worth  a  combined  value  of  at  least  $5  million  during  the 
preceding  fiscal  year.5  In  addition,  the  contractor  must  have  achieved  a 
small  disadvantaged  business  subcontracting  participation  rate  of  at  least 
5  percent  during  the  preceding  fiscal  year.  Participation  in  the  Test 
Program  is  voluntary  and,  as  shown  in  table  2,  there  are  currently  12 
participants. 


sThe  Carl  Levin  and  Howard  P.  “Buck”  McKeon  National  Defense  Authorization  Act  for 
Fiscal  Year  2015,  Pub.  L.  No.  11 3-291 ,  §  821  (b)(2)  raises  the  monetary  threshold  to  $1 00 
million. 
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Table  2:  Test  Program  Participants,  Year  of  Entry,  and  Participation  Level,  as  of 
Fiscal  Year  2015 

Participant 

Entry  date 
(fiscal  year) 

Level  of 
participation 

BAE  Systems 

2007 

Division 

The  Boeing  Company3 

1991 

Corporate 

GE  Aviation 

1991 

Division 

General  Dynamics/C4  Systems,  Advanced 
Information  Systems  and  Information  Technology 

2007 

Division 

Hamilton  Sundstrand  Corporation 

1996 

Division 

Harris  Corporation  Government  Communications 
Systems 

1996 

Division 

L-3  Communications  CSB  Sector 

2006 

Division 

Lockheed  Martin  Corporation 

2004 

Corporate 

Northrop  Grumman  Electronic  Systems 

1996 

Division 

Pratt  &  Whitney  Government  Division 

2002 

Division 

Raytheon  Company 

1999 

Corporate 

Sikorsky  Aircraft  Corporation 

1996 

Division 

Sources:  DOD  and  Defense  Contract  Management  Agency  officials.  |  GAO-16-27 

aThe  Boeing  Company  ceased  participation  in  2007,  but  active  contracts  awarded  before  that  time  are 
still  reported  through  the  Test  Program. 


According  to  data  in  the  Federal  Procurement  Data  System — Next 
Generation,  the  federal  government’s  contract  reporting  system,  8  of 
these  contractors  are  ranked  among  the  top  10  U.S.  defense  contractors, 
based  on  contract  dollars  obligated  in  fiscal  year  2014. 

According  to  DOD  officials,  the  DOD  Office  of  Small  Business  Programs 
(OSBP)  is  responsible  for  overseeing  the  Test  Program,  but  delegates  the 
management  and  oversight  of  comprehensive  subcontracting  plan  annual 
negotiations  and  performance  evaluation  to  the  Defense  Contract 
Management  Agency  (DCMA).  DCMA  is  responsible  for  reviewing  and 
approving  Test  Program  participants’  proposed  comprehensive 
subcontracting  goals  and  initiatives  to  ensure  that  they  are  challenging 
yet  realistic.  DCMA  is  also  responsible  for  reviewing  the  achievements  of 
the  participants  at  the  end  of  the  fiscal  year  and  rating  their  performance. 
The  Test  Program  does  not  have  an  overall  measure  for  demonstrating 
success  in  creating  small  business  subcontracting  opportunities;  instead, 
metrics  are  established  for  each  individual  participant.  The  performance 
of  participants  in  the  Test  Program  is  measured  by  their  achievement  of 
negotiated  goals  and  initiatives. 
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Congress  has  extended  the  program  eight  times  but  has  not  made  the 
program  permanent  in  part  because  of  a  lack  of  data  on  program 
performance.  The  latest  extension  was  in  the  National  Defense 
Authorization  Act  for  Fiscal  Year  2015  in  which  Congress  temporarily 
extended  the  Test  Program  to  December  31,  2017,  and  made  a  number 
of  amendments  to  the  program.  These  amendments  included,  among 
other  things:  an  increase  in  the  eligibility  requirements;  additional 
reporting  requirements  on  subcontracting  activities  and  costs  to  increase 
visibility  at  the  contract,  program,  and  military  department  levels;  and  an 
additional  consequence  for  failure  to  make  good  faith  efforts  to  comply 
with  the  program.6 

DOD  commissioned  reviews  of  the  Test  Program  from  two  consulting 
groups  in  2002,  2007,  and  2013.  These  reviews  examined  the 
performance  of  Test  Program  participants  against  their  goals  and 
initiatives  and  whether  administrative  savings  had  been  achieved.7 
According  to  DOD  officials,  DOD  has  not  released  these  reviews  publicly 
or  submitted  them  to  the  Congress.  All  of  the  reviews  found  that  the 
program  resulted  in  administrative  costs  savings  and  enhanced 
subcontracting  opportunities  for  small  businesses. 

Some  in  the  small  business  community  have  publicly  raised  concerns 
about  the  Test  Program.  The  primary  concern  is  that  the  lack  of  data 
available  to  evaluate  the  program  precludes  a  determination  of  the 
program’s  effectiveness  and  impact  on  small  businesses.  Specifically, 
some  small  business  advocates  believe  that  more  data  are  needed  to 
evaluate  whether  the  program  has  resulted  in  more  awards  in  areas  such 
as  innovative  technology  research  that  traditionally  have  not  been  made 
available  to  small  businesses. 


sThe  Carl  Levin  and  Howard  P.  “Buck”  McKeon  National  Defense  Authorization  Act  for 
Fiscal  Year  2015,  Pub.  L.  No.  1 1 3-291 ,  §  821 . 

7The  2002  and  2007  reviews  combined  to  offer  comprehensive  data  about  the  results  of 
the  Test  Program  through  fiscal  year  2005.  Therefore,  we  selected  fiscal  years  2006 
through  201 3,  the  last  year  for  which  full  data  were  available  for  our  audit,  for  our  analysis 
of  program  results.  The  2013  DOD-commissioned  review  reported  only  a  summary  of 
program  results. 
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Test  Program  Allows 
Participants  and  DOD 
to  Avoid  Millions  in 
Administrative  Costs 
and  May  Provide 
Additional  Benefits 


Prior  DOD  reviews  estimated  that  use  of  the  Test  Program  resulted  in  the 
avoidance  of  millions  of  dollars  in  administrative  costs  for  both 
participants  and  DOD  and  may  offer  other  benefits.  Significant  one-time 
administrative  costs  could  result  if  the  Test  Program  were  canceled  or 
allowed  to  expire  and  comprehensive  subcontracting  plans  could  no 
longer  be  used.  Test  Program  participants  and  DOD  officials  we 
interviewed  stated  that  the  use  of  comprehensive  subcontracting  plans 
results  in  benefits  other  than  administrative  cost  avoidance  as  well. 
However,  DOD  has  not  taken  action  to  address  the  program’s  status. 


Test  Program  Reduces 
Administrative  Costs  for 
Participants  and  DOD 


DOD’s  reviews  estimated  that  the  program  resulted  in  millions  of  dollars 
in  cost  avoidance  for  the  program’s  participants.  According  to  these 
reviews,  cost  avoidance  is  enabled  primarily  by  the  use  of  a  single 
comprehensive  subcontracting  plan  for  multiple  contracts  rather  than  an 
individual  subcontracting  plan  for  each  contract.  Our  examination  of  the 
methodologies  and  data  used  in  these  reviews,  as  well  as  our  own 
analysis  of  more  recent  data  on  contracts  covered  by  comprehensive 
subcontracting  plans,  support  these  conclusions.  Table  3  shows  the  total 
estimated  annual  administrative  costs  avoided  by  the  participants, 
according  to  the  DOD  reviews. 


Table  3:  Test  Program  Total  Estimated  Annual  Administrative  Costs  Avoided  as 
Determined  by  DOD  Reviews 


Number  of 

Total  estimated  annual 

Year  of  review 

participating  firms 

administrative  costs  avoided3 

2002 

20 

Between  $2.3  and  $4.6  million 

2007 

13 

Between  $6.8  and  $13.7  million 

2013 

12 

$18.5  million 

Source:  DOD.  |  GAO-16-27 

aFigures  are  in  then-year  dollars.  The  2002  and  2007  reviews  used  estimated  ranges  in  their 
calculations  for  avoided  costs,  while  the  2013  review  incorporated  costs  collected  from  the 
participants. 

Some  of  the  Test  Program  participant  officials  we  interviewed  explained 
that  they  have  not  quantified  their  administrative  cost  avoidance  under  the 
program;  however,  they  stated  that  savings  likely  accrued  from 
negotiating  a  single  plan  for  multiple  contracts  as  opposed  to  individual 
plans.  One  participant  official  stated  that  utilizing  comprehensive 
subcontracting  plans  also  allows  them  to  use  fewer  people  for  the 
administrative  tasks  of  developing  and  monitoring  subcontracting  plans. 
Certain  participants  noted  that  the  resources  that  would  have  gone  to  the 
development  and  administration  of  individual  plans  can  be  used  instead 
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for  increased  small  business  outreach  activities  such  as  more  small 
business  forums.  Expansion  of  the  Test  Program  from  division-level  to 
company-level  could  increase  the  cost  avoidance  for  certain  participants, 
but  officials  from  one  division-level  participant  we  interviewed  stated  that 
the  continued  test  status  of  the  program  served  to  inhibit  that  expansion. 
The  2007  DOD  review  also  found  the  cost  avoided  by  Test  Program 
participants  and  DOD  would  increase  substantially  if  more  prime 
contractors  used  the  program,  but  that  this  was  unlikely  due  to  their 
reluctance  to  enter  the  program  because  of  its  test  status. 

In  addition  to  the  costs  avoided  by  program  participants,  DOD  also  may 
benefit  from  avoiding  administrative  costs  as  a  result  of  the  Test  Program. 
The  2007  DOD  review  found  that  DOD  benefited  from  negotiating, 
administering,  and  monitoring  consolidated  small  business  subcontracting 
plans  rather  than  multiple  individual  plans.  For  example,  for  fiscal  year 
2005,  the  review  estimated  that  the  administrative  cost  avoided  by  DOD 
under  the  Test  Program  was  at  least  $45  million.  Defense  officials  stated 
that  additional  participation  in  the  Test  Program  by  other  prime 
contractors  could  increase  the  department’s  cost  avoidance,  but  stated 
that  they  are  not  considering  new  participants  for  the  program  due  to  its 
test  status. 


Administrative  Costs  to 
Convert  Existing  Test 
Program  Plan  Contracts  to 
Individual  Contracting 
Plans  Could  Be 
Substantial 


DOD’s  2013  review  noted  that  the  program  participants  at  that  time  had 
about  7,000  contracts  and  subcontracts  that  would  require  individual 
subcontracting  plans  if  the  Test  Program  did  not  exist.  The  estimated  cost 
to  convert  the  contracts  to  individual  subcontracting  plans  if  the  Test 
Program  were  canceled  or  allowed  to  expire  was  $21.7  million.  The 
review  also  noted  that  the  combined  cost  for  the  participants  to  prepare, 
submit  and  negotiate  their  comprehensive  subcontracting  plans  on  an 
annual  basis  under  the  Test  Program  was  approximately  $660,000.  In 
addition,  most  of  the  current  participants  have  been  in  the  Test  Program 
for  at  least  10  years;  two  have  been  in  the  program  since  its  inception  25 
years  ago.  As  a  result,  converting  from  comprehensive  subcontracting 
plans  to  individual  subcontracting  plans  may  require  participants  to  add 
personnel  and  change  their  current  administrative  systems.  One 
participant  noted  that  reverting  to  individual  subcontracting  plans  would 
require  an  additional  44  employees  to  handle  the  additional  administrative 
workload  and  an  estimated  $2  million  in  system  changes. 


Our  analysis  of  similar  data  supports  these  estimates.  We  analyzed  data 
from  DCMA  for  contracts  that  were  active  as  of  March  2015  for  6  of  the 
12  Test  Program  participants,  including  those  with  the  largest  number  of 
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contracts,  and  found  a  total  of  3,299  contracts  that  would  require 
individual  subcontracting  plans  in  the  absence  of  the  Test  Program.  Using 
estimates  provided  by  DOD  for  the  cost  and  number  of  hours  required  to 
develop  an  individual  subcontracting  plan,  we  determined  that  these  six 
participants  would  need  to  spend  between  $6.3  million  and  $9.5  million  to 
convert  existing  contracts  to  individual  subcontracting  plans.  One 
participant  we  interviewed  estimated  that  for  fiscal  year  2013,  the  cost  to 
convert  all  of  the  contracts  under  the  Test  Program  to  individual  plans 
would  have  been  approximately  $6.6  million.  The  cost  to  this  participant 
to  prepare,  review,  and  negotiate  its  fiscal  year  2013  comprehensive 
subcontracting  plan  was  $159,000. 

As  is  the  case  with  the  Test  Program  participants,  DOD  would  likely 
experience  an  increase  in  costs  for  negotiating,  administering,  and 
monitoring  individual  small  business  subcontracting  plans  if  the  program 
were  to  end.  Defense  officials  noted  that  DOD  would  have  to  hire 
additional  personnel  in  DCMA  and  the  military  services  to  manage  the 
thousands  of  additional  individual  subcontracting  plans.  These 
personnel — which  would  include  contracting  officers,  small  business 
specialists,  and  cost  analysts — all  require  specific  levels  of  expertise  and 
training  depending  on  the  cost  and  complexity  of  the  contract.  According 
to  DOD  officials,  it  would  be  challenging  to  provide  the  amount  of 
resources  required  with  the  limited  acquisition  workforce  DOD  employs 
and  available  budgets  for  hiring  new  personnel. 


Test  Program  May  Result 
in  Other  Benefits  for 
Participants,  DOD,  and 
Small  Busineses  but  DOD 
Has  Not  Taken  Action  to 
Address  Program  Status 


According  to  some  of  the  Test  Program  participants  we  interviewed,  the 
use  of  comprehensive  subcontracting  plans  as  opposed  to  individual 
subcontracting  plans  also  provides  additional  non-monetary  benefits.  For 
example,  certain  participants  stated  that  they  benefit  from  the  approach  of 
consolidating  small  business  subcontracting,  as  it  allows  them  to  consider 
or  leverage  small  business  subcontracting  opportunities  across  the  whole 
organization  rather  than  on  a  contract-by-contract  basis.  Some 
participants  stated  that  as  a  result  of  this  approach  they  have  increased 
small  business  subcontracting  outside  of  their  defense  contracts  by 
utilizing  the  same  small  businesses  cultivated  under  the  Test  Program  for 
other  work.  All  of  the  participants  we  interviewed  noted  that  the  issue  of 
small  business  subcontracting  has  greater  visibility  and  awareness  with 
corporate  leadership  as  a  result  of  the  Test  Program  and  that  their  entities 
might  be  less  inclined  to  award  subcontracts  to  small  businesses  in  its 
absence. 
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DOD  officials  agreed  that,  in  addition  to  administrative  cost  avoidance, 
the  Test  Program  gives  them  leverage  they  may  not  have  in  negotiating 
individual  contracting  plans.  The  ability  to  require  the  contractor  to  look  for 
opportunities  across  multiple  contracts  increases  the  likelihood  of  small 
business  outreach  and  subcontract  awards.  Officials  stated,  however,  that 
there  are  some  concerns  within  the  department  about  the  Test  Program. 
For  example,  they  stated  that  reporting  of  small  business  subcontracting 
activity  at  a  division  or  corporate  level  reduces  the  department’s  visibility 
into  small  business  activity  on  a  contract  or  programmatic  level.  In 
addition,  DOD  officials  said  that  the  continuing  test  status  of  the  program 
has  made  it  difficult  for  OSBP  and  DCMA  to  develop  the  policies  and 
guidance  they  believe  are  needed  to  enhance  program  activities. 

The  2007  review  strongly  recommended  that  DOD  work  with  Congress  to 
make  the  Test  Program  permanent.  According  to  DOD  officials,  DOD  has 
not  acted  on  this  recommendation,  by  drafting  a  legislative  proposal  or 
taking  other  actions,  primarily  due  to  concerns  that  the  program,  at  least 
as  structured  prior  to  the  latest  legislative  changes,  may  limit  visibility  into 
small  business  subcontracting  activities  for  individual  programs  or 
contracts.  However,  implementation  of  the  increased  reporting 
requirements  legislated  in  the  National  Defense  Authorization  Act  for 
Fiscal  Year  2015  may  alleviate  these  concerns  as  they  include  reporting 
on  small  business  subcontracting  activities  at,  among  other  things,  the 
contract  and  major  defense  acquisition  program  level.8  Working  with 
Congress  to  address  the  program’s  status,  for  example  by  providing 
information  on  the  effectiveness  of  the  Test  Program  as  identified  in  the 
three  DOD-commissioned  reviews  and  our  analysis,  could  help  eliminate 
the  uncertainty  associated  with  the  program. 


8The  Carl  Levin  and  Howard  P.  “Buck”  McKeon  National  Defense  Authorization  Act  for 
Fiscal  Year  2015,  Pub.  L.  No.  1 1 3-291 ,  §  821  (b), 
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Test  Program 
Enhanced  Small 
Business 
Opportunities, 
Though  Participant 
Performance  Varied 


In  reviewing  participants’  performance  in  the  Test  Program,  DCMA  looks 
at  whether  participants  have  achieved  the  negotiated  initiatives  and  goals 
associated  with  their  comprehensive  subcontracting  plans,  and  assigns 
an  overall  rating  for  that  fiscal  year’s  performance.  Our  analysis  found 
that  87  percent  of  DCMA’s  reviews  indicated  that  participants  made 
acceptable  progress  on  their  initiatives  to  enhance  small  business 
opportunities,  but  that  participants’  performance  in  meeting  their 
negotiated  subcontracting  goals  varied.  Our  analysis  also  found  that  while 
participants  did  not  always  meet  individual  goals,  their  aggregate 
performance  resulted  in  subcontract  awards  to  small  business  that 
exceeded  aggregate  goals  by  approximately  $5.4  billion  from  fiscal  years 
2006  through  2013.  When  compared  to  the  performance  of  contractors 
not  participating  in  the  program  (nonparticipants),  Test  Program 
participants  have  lower  percentages  of  dollars  awarded  to  small 
businesses.  Participants  we  interviewed  explained  that  this  is  due,  among 
other  things,  to  the  nature  of  the  contracts  they  perform  and  the  way  the 
Test  Program  tracks  achievement  of  goals.  Finally,  DCMA’s  annual 
performance  reviews,  which  assess  achievements  against  both 
negotiated  initiatives  and  goals,  have  been  largely  positive. 


Implementation  of 
Initiatives  Enhanced 
Subcontracting 
Opportunities  for  Small 
Businesses 


Participants  made  acceptable  progress,  as  assessed  by  DCMA,  on  their 
negotiated  initiatives  in  74  of  the  85  reviews  we  analyzed — an  87  percent 
success  rate.  Our  analysis  indicates  that  completion  of  these  initiatives 
has  resulted  in  tangible  subcontracting  opportunities  for  small  businesses. 
DOD  officials  stated  the  achievement  of  these  initiatives  is  not  tied  to  any 
one  contract  within  the  comprehensive  subcontracting  plan;  any  of  the 
defense  work  being  subcontracted  by  a  prime  contractor  can  be  used  to 
meet  the  initiative.  According  to  a  participant  and  DOD  officials,  the 
initiatives  offer  a  distinct  advantage  over  individual  subcontracting  plans. 
Examples  of  initiatives  include: 

•  redirecting  subcontracts  from  large  businesses  to  small  businesses; 

•  targeting  increased  small  business  subcontracts  in  technical  fields 
such  as  integrated  circuits,  computer  information  technology  products, 
and  engineering  and  technical  services;  and 
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•  participating  in  the  Small  Business  Innovation  Research  (SBIR)  and 
Mentor-Protege  Programs.9 

Some  of  these  initiatives  target  the  socioeconomic  categories  of  small 
businesses  for  which  the  participants  had  the  most  difficulty  achieving 
performance  goals.  For  example,  one  participant  failed  to  meet  its 
subcontracting  goals  for  the  small  disadvantaged  business  category  over 
a  3-year  period.  For  each  of  these  years  the  participant  had  an  initiative  to 
increase  performance  in  that  category. 

According  to  our  analysis  of  DCMA  reviews,  participants  are  generally 
successful  at  achieving  their  initiatives.  In  our  review  period  of  fiscal  years 
2006  through  2013,  we  found 

•  16  initiatives  resulted  in  the  redirect  of  approximately  $93  million  in 
subcontracts  from  large  business  to  small  businesses; 

•  a  72-percent  success  rate  in  participants  meeting  milestones  for 
increasing  small  business  subcontracts  in  targeted  industries; 

•  24  mentor-protege  initiatives  resulted  in  61  new  mentor-protege 
relationships  between  participants  and  small  businesses;  and 

•  1 1  SBIR  projects  that  identified  83  new  small  business  suppliers. 

The  2013  DOD  review  estimated  that  initiatives  completed  by  Test 
Program  participants  could  amount  to  as  much  as  $1 .8  billion  per  year  in 
increased  small  business  opportunities,  and  that  participants  spent  as 
much  as  $5.5  million  annually  on  small  business  subcontracting 
enhancements  and  initiatives. 


Participants’  Performance  Our  analysis  found  that  Test  Program  participants’  performance  in 
in  Meeting  Goals  Varies  meeting  their  negotiated  subcontracting  goals  varied.  The  Test  Program 


9The  Small  Business  Innovation  Research  Program  was  established  in  1982  for  use  by 
civilian  and  defense  agencies  to  stimulate  technological  innovation  and  utilize  small 
businesses  to  meet  federal  research  and  development  needs.  Small  Business  Innovation 
Development  Act  of  1 982,  Pub.  L.  No.  97-219,  §  2.  DOD’s  Mentor-Protege  Program  was 
established  in  1991  to  provide  incentives  for  major  DOD  contractors  to  serve  as  mentors 
to  small  disadvantaged  businesses  thereby  increasing  small  business  capabilities  and 
participation  in  subcontracts.  National  Defense  Authorization  Act  for  Fiscal  Year  1991, 
Pub.  L.  No.  101-510,  §  831. 


Page  12 


GAO-16-27  Small  Business  Subcontracting 


does  not  have  overall  small  business  subcontracting  goals  similar  to 
those  that  DOD  negotiates  annually  with  the  Small  Business 
Administration,  but  does  have  goals  on  a  contractor-by-contractor  basis.10 
The  goals,  as  well  as  the  achievements,  for  small  business 
subcontracting  are  expressed  in  terms  of  (1)  the  total  dollars  awarded  to 
small  businesses,  and  (2)  the  dollars  awarded  to  small  business  as  a 
percent  of  total  subcontract  dollars. 

DCMA  is  responsible  for  annually  negotiating  and  approving  Test 
Program  participants’  dollar  and  percentage  goals  to  ensure  that  they  are 
realistic  and  challenging.  In  all  the  negotiation  support  memorandums  we 
reviewed,  DCMA  attempted  to  negotiate  with  the  participants  to  establish 
challenging  goals.  According  to  DCMA  documentation,  this  was 
accomplished  by,  for  example,  analyzing  a  participant’s  5-year 
subcontracting  performance  trends  and  comparing  them  to  submitted 
goals,  or  assessing  a  participant’s  documented  support  of  its  proposed 
goals  to  determine  if  a  goal  higher  than  that  presented  by  the  participant 
was  realistic. 

As  shown  in  figure  1,  the  number  of  Test  Program  participants  that  met  or 
exceeded  goals  varied  by  fiscal  year. 


10DOD  and  the  federal  government  annually  negotiate  with  the  Small  Business 
Administration  their  small  business  subcontracting  goals.  In  fiscal  year  2013,  for  example, 
the  DOD  overall  small  business  goal  was  36.7  percent  and  the  federal  government’s 
overall  goal  was  36  percent. 
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Figure  1 :  Number  of  Test  Program  Participants  that  Met  or  Exceeded  Their  Small  Business  Subcontracting  Dollar  and 
Percentage  Goals,  Fiscal  Years  2006  through  2013 

Number  of  participants 

15 

14 


Source:  GAO  analysis  of  data  in  Defense  Contract  Management  Agency  annual  performance  reviews.  |  GAO-16-27 

Note:  The  number  of  Test  Program  participants  may  change  from  year  to  year  due  to  contractors 
entering  and  exiting  the  program  as  well  as  mergers  and  acquisitions. 


According  to  some  participants  we  interviewed,  there  are  a  number  of 
reasons  that  individual  participants  may  not  meet  their  annual  goals.  One 
reason  is  that  the  goals  are  intended  to  be  “stretch  goals”  beyond  what 
the  participants  are  confident  they  know  can  be  achieved.  Our  analysis 
also  shows  that  when  participants  failed  to  meet  a  goal  they  generally 
missed  the  goal  by  a  small  percentage.  For  example,  in  fiscal  year  201 1 , 
the  eight  participants  who  missed  their  goals  all  did  so  by  less  than  10 
percent  and  in  two  cases  by  less  than  1  percent.  Some  participants  also 
stated  that  their  inability  to  renegotiate  goals  to  account  for  changes  in  the 
dollars  available  for  subcontracting  may  also  negatively  affect  their 
achievements.  DCMA  officials  said  that  they  generally  prefer  not  to  allow 
for  renegotiation,  as  it  is  a  time  consuming  process. 

When  examining  the  performance  of  Test  Program  participants  as  a 
group,  we  found  that  they  exceeded  their  combined  goals  for 
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subcontracting  dollars  awarded  to  small  businesses  in  each  year  of  our 
review  period,  as  shown  in  figure  2,  for  a  total  of  approximately  $5.4 
billion  above  the  combined  goals  from  fiscal  years  2006  through  2013. 


Figure  2:  Test  Program  Dollar  Achievements  in  Comparison  to  Dollar  Goals  for 
Small  Business  Subcontracting,  Fiscal  Years  2006  through  2013 
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For  the  goal  of  percentage  of  total  subcontracting  dollars  awarded  to 
small  business,  we  found  that,  at  an  aggregate  level,  participants 
exceeded  goals  in  3  of  the  8  fiscal  years  we  reviewed  and  missed 
achieving  those  goals  by  less  than  4  percent  in  every  other  fiscal  year. 
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Various  Reasons  Given  for 
Disparity  in  Percentage  of 
Subcontracting  Dollars 
Awarded  to  Small 
Businesses  by 
Participants  Compared  to 
Other  Defense 
Contractors 


In  our  discussion  with  one  small  business  advocate,  one  of  the  criticisms 
of  the  Test  Program  raised  was  that  participants  generally  awarded  a 
lower  percentage  of  their  subcontracting  dollars  to  small  businesses  than 
nonparticipants.  Our  analysis,  as  shown  in  figure  3,  shows  this  to  be  the 
case. 


Figure  3:  Comparison  of  Test  Program  Participants  and  Nonparticipants  for 
Percentage  of  Subcontracting  Dollars  Awarded  to  Small  Businesses,  Fiscal  Years 
2006  through  2013 
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For  Test  Program  participants,  the  percentage  of  subcontract  dollars 
awarded  to  small  businesses  ranged  from  a  low  of  around  25  percent  in 
fiscal  year  201 1  to  a  high  of  around  29  percent  in  fiscal  year  2008. 
However,  for  nonparticipants  the  percentage  was  a  low  of  around  33 
percent  in  fiscal  year  2008,  and  a  high  of  nearly  43  percent  in  fiscal  year 
2006. 

We  found  that  over  the  8-year  review  period,  Test  Program  participants 
also  experienced  less  success  in  achieving  their  negotiated  goals  for  the 
various  socioeconomic  categories  of  small  businesses  when  compared  to 
nonparticipants’  achievements.  Test  Program  participants  consistently 
awarded,  on  the  basis  of  percentage  of  subcontract  dollars  awarded, 
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lower  percentages  than  nonparticipants.  The  only  fiscal  year  that  Test 
Program  participants’  awarded  higher  amounts  than  nonparticipants  was 
in  fiscal  year  2008  and  then  only  for  2  of  the  5  subcategories  measured. 

According  to  some  participants  we  interviewed,  there  are  a  number  of 
factors  that  can  adversely  affect  their  ability  to  achieve  their  annual 
percentage  and  dollar  subcontracting  goals.  Examples  of  these  factors 
include  funding  changes  that  affect  the  value  of  a  contract  or  other 
changes  to  contracts  that  affect  the  amounts  available  for  subcontracting. 
These  factors  are  not  unique  to  Test  Program  participants  and  could 
apply  to  any  contractor  involved  in  federal  contracting;  however,  there  are 
some  factors  that  may  have  a  disproportionate  effect  on  Test  Program 
participants.  These  factors  include  the  following: 

•  Test  Program  participants  represent  some  of  DOD’s  largest 
prime  contractors.  As  a  result,  many  of  the  contracts  they  receive 
from  DOD  are  large,  technically  complex,  and  include  a  scope  of  work 
that  may  require  teaming  arrangements  with  other  large  businesses, 
reducing  the  dollars  available  for  subcontracting  to  small  businesses. 
For  example,  the  F-35  Joint  Strike  Fighter  program  has  a  prime 
contract  with  Lockheed  Martin  Corporation.  For  this  highly  complex 
and  technical  work,  Lockheed  Martin  has  subcontracted  significant 
portions  of  the  work  to  Northrup  Grumman  Corporation  and  BAE 
Systems,  neither  of  which  is  considered  a  small  business. 

•  Test  Program  participants  can  only  count  certain  small  business 
subcontracting  activities  toward  their  goals.  While  small 
businesses  may  be  involved  at  some  level  of  subcontracting — either 
through  a  direct  award  from  the  prime  contractor  as  a  first-tier 
subcontractor  or  as  a  second-tier  subcontractor  through  an  award 
from  a  first-tier  subcontractor — the  ability  of  a  Test  Program 
participant  to  count  small  business  subcontracting  dollars  towards 
their  performance  goals  is  limited.  According  to  Test  Program 
participants  and  DOD  officials  we  interviewed,  federal  acquisition 
regulations  require  that  goals  for  small  business  subcontracting  be 
based  on  the  prime  contractor’s  first-tier  subcontracts.  This 
government-wide  practice  does  not  permit  prime  contractors  to  report 
subcontracting  below  the  first-tier  for  purposes  of  demonstrating  the 
achievement  of  small  business  subcontracting  goals.  Therefore, 
participants  can  only  take  credit  for  awards  made  at  the  first-tier  of 
subcontracting.  For  example,  for  the  F-35  program  Lockheed  Martin 
has  approximately  500  first-tier  suppliers  and  1,250  second-tier 
suppliers.  Lockheed  Martin  does  not  receive  credit  for  small  business 
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subcontract  achievements  below  the  first-tier,  which  could  represent  a 
significant  amount  of  small  business  subcontracting. 

•  If  a  small  business  subcontractor  grows  into  a  large  business,  its 
subcontracts  are  no  longer  counted  toward  participants’  goals. 

The  determination  of  a  subcontractor’s  size  as  small  for 
subcontracting  purposes  is  set  on  the  date  that  it  self-certifies  that  it  is 
small  for  the  subcontract  and  is  not  typically  revisited  for  the  duration 
of  the  contract.  Some  participants  we  interviewed  stated  that,  for  their 
comprehensive  subcontracting  plans,  the  size  determination  is 
revisited  when  assessing  progress  against  annual  goals.  For 
example,  if  a  small  business  becomes  successful  and  wins  contracts 
that  grow  it  into  a  larger  business  during  the  comprehensive 
subcontracting  plan’s  annual  assessment  period,  a  Test  Program 
participant  cannot  count  those  subcontracting  dollars  toward  the 
attainment  of  its  small  business  goals.  For  instance,  if  a  participant 
subcontracts  work  to  a  veteran-owned  small  business  which  then  wins 
contracts  from  other  sources  and  grows  beyond  the  small  business 
size  standards,  those  subcontract  dollars  are  excluded  from  the 
calculation  of  goal  attainment. 

•  Make  or  buy  decision.  Participants  may  make  decisions  to  produce 
components  in-house  rather  than  buying  them  from  a  supplier  in  order 
to  reduce  the  overall  contract  cost.  This  decision  thus  reduces  the 
amount  of  subcontracting  dollars  available  to  small  businesses. 


Overall  Ratings  on 
Participants’  Performance 
Are  Generally  Positive 


To  achieve  another  perspective  on  Test  Program  performance,  we 
analyzed  DCMA  performance  ratings  for  the  individual  participants.  In  our 
review  of  85  annual  performance  reviews  for  fiscal  years  2006  through 
2013,  Test  Program  participants  generally  received  positive  annual 
performance  ratings  from  DCMA.  DCMA  is  responsible  for  annually 
evaluating  the  performance  of  Test  Program  participants  and  making 
recommendations  as  to  their  continued  participation  in  the  program. 

When  evaluating  the  participants,  DCMA  is  to  consider  their  performance 
on  both  goals  and  initiatives  and,  if  they  failed  to  achieve  these,  whether 
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the  participants  made  a  “good  faith  effort”  to  do  so.11  Participants  receive 
an  annual  overall  program  rating  ranging  from  “Unsatisfactory”  to 
“Outstanding.”  Participants  that  do  not  receive  at  least  an  “Acceptable” 
rating  are  required  to  submit  to  DCMA  a  detailed  corrective  action  plan  to 
account  for  and  improve  on  known  deficiencies. 

For  example,  for  fiscal  year  201 3,  the  12  participating  firms  received  the 
following  ratings: 

•  Five  participants  received  an  Outstanding  rating — meaning  they 
generally  exceeded  the  annual  negotiated  small  business  goals  and 
two  additional  socioeconomic  category  goals  and  had  exceptional 
success  with  numerous  specific  initiatives  to  assist,  promote,  and 
utilize  small  businesses. 

•  Four  participants  received  a  Highly  Successful  rating — meaning  they 
generally  met  or  exceeded  negotiated  goals,  including  three  small 
business  categories,  and  had  moderate  success  with  some  initiatives 
to  assist,  promote,  and  utilize  small  businesses. 

•  Three  participants  received  an  Acceptable  rating — meaning  they 
generally  demonstrated  a  good  faith  effort  to  meet  all  of  their  annual 
subcontracting  goals  and  provided  reasonable  and  supportable 
explanations  why  certain  goals  could  not  be  met. 

In  the  period  we  reviewed,  fiscal  years  2006  through  2013,  no  participants 
received  a  rating  of  “Unsatisfactory”  and  three  participants  received  a 
rating  of  “Marginal.”  Also,  none  of  the  participants  we  reviewed  in  the 
period  were  found  by  DCMA  to  not  have  made  a  “good  faith  effort”  in 
seeking  to  meet  their  goals,  no  matter  what  their  overall  rating.  According 
to  DCMA  annual  performance  documentation,  the  “Marginal”  ratings  were 
assigned  generally  because  the  participants  were  deficient  in  meeting  key 


1  According  to  the  Small  Business  Administration,  a  prime  contractor  can  demonstrate  that 
it  has  made  a  good  faith  effort  in  several  ways  including  by  (1 )  meeting  its  dollar  and 
percentage  goals;  (2)  overachieving  in  some  small  business  categories  to  make  up  for  an 
underachievement  in  other  categories;  or  (3)  performing  one  of  a  variety  of  things  (e.g., 
participate  in  a  formal  mentor-protege  program  or  break  out  contract  work  items  into 
smaller  units  to  facilitate  small  business  participation).  FAR  Subpart  1 9.701  defines  failure 
to  make  a  good  faith  effort  to  comply  with  the  subcontracting  plan  to  mean  “willful  or 
intentional  failure  to  perform  in  accordance  with  the  requirements  of  the  subcontracting 
plan,  or  willful  or  intentional  action  to  frustrate  the  plan.” 
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subcontracting  plan  elements,  or  the  contractor  failed  to  satisfy  one  or 
more  requirements  of  a  corrective  plan  from  a  prior  review.  Two  of  these 
participants  achieved  ratings  at  the  “Acceptable”  level  or  above  in 
subsequent  reporting  periods  and  one  voluntarily  exited  the  program. 

DOD  officials  said  they  have  never  terminated  a  participant  from  the 
program,  but  that  some  have  voluntarily  exited  the  program.  DOD  officials 
stated  that  ratings  are  taken  into  consideration  by  both  DOD  and 
participants  when  negotiating  future  participation  in  the  program  and  may 
be  taken  into  consideration  as  part  of  the  determination  of  past 
performance  by  contracting  officers  in  awarding  future  government 
contracts.  One  participant  representative  highlighted  that  the  company 
uses  its  high  ratings  as  a  marketing  tool  to  attract  small  businesses. 


Conclusions 


Because  of  its  large  contracting  operations,  DOD  is  critical  to  the  success 
of  federal  programs  designed  to  provide  opportunities  for  small 
businesses.  The  Test  Program  is  aimed  at  enhancing  these  opportunities 
and  reducing  participants’  administrative  costs.  The  evidence  collected  by 
DOD  and  our  analysis  of  that  evidence  indicates  that  the  program  has 
achieved  these  goals.  The  use  of  comprehensive  subcontracting  plans 
allows  both  DOD  and  Test  Program  participants  to  avoid  millions  of 
dollars  in  administrative  costs  and  has  led  to  demonstrable 
enhancements  in  small  business  subcontracting  opportunities,  thereby 
meeting  the  criteria  established  by  Congress.  While  there  may  be 
concerns  about  the  visibility  of  small  business  subcontracting  on 
particular  contracts  or  programs,  these  concerns  may  diminish  when 
legislative  changes  made  to  the  program  as  part  of  the  fiscal  year  201 5 
National  Defense  Authorization  Act  are  implemented.  Given  that  the  Test 
Program  has  been  in  existence  for  25  years,  and  therefore  has  become  a 
de  facto  permanent  program  for  both  DOD  and  participants,  termination 
of  the  program  in  favor  of  individual  subcontracting  plans  would  likely 
require  substantial  increases  in  manpower  and  fiscal  resources.  However, 
DOD  has  not  acted  on  a  2007  review  recommendation  to  work  with 
Congress  to  make  the  Test  Program  permanent.  Continually  extending 
the  program  rather  than  making  it  permanent  creates  uncertainty  among 
participants  and  DOD,  inhibiting  the  expansion  of  the  program  by  some 
participants,  the  inclusion  of  new  participants,  and  the  formulation  of  DOD 
policies  and  additional  guidance  that  could  enhance  the  program’s 
results.  Working  with  Congress  to  address  the  program’s  status,  for 
example  by  providing  information  on  the  effectiveness  of  the  Test 
Program  as  identified  in  the  three  DOD-commissioned  reviews  and  our 
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analysis,  could  help  eliminate  the  uncertainty  associated  with  the 
program. 


Agency  Comments 
and  Our  Evaluation 


In  written  comments,  DOD  did  not  concur  with  the  draft  report’s 
recommendation  to  draft  a  legislative  proposal  to  make  the  program 
permanent  and  questioned  the  finding  related  to  the  Test  Program 
enhancing  small  business  subcontracting  opportunities.  The  department 
agreed  to  work  with  Congress  to  determine  the  status  of  the  Test 
Program.  Given  DOD’s  disagreement,  we  added  a  matter  for 
Congressional  consideration  to  this  report  and  modified  the 
recommendation  as  discussed  below.  DOD’s  comments  are  provided  in 
full  in  appendix  II. 


Related  to  the  finding  on  enhancing  small  business  subcontracting 
opportunities,  with  its  comments  on  our  draft  report,  DOD  provided  a 
chart  that  shows  a  decrease  in  the  percentage  of  subcontract  dollars 
awarded  to  small  business  by  Test  Program  participants  from  fiscal  years 
1996  through  2014.  The  department  stated  in  its  response  that  “[t]his 
trend  shows  that  the  current  practices  associated  with  the  [Test  Program] 
do  not  enhance  opportunities  for  small  businesses.”  We  do  not  agree  with 
that  assessment  for  a  number  of  reasons. 


•  While  the  percentage  of  subcontracting  dollars  awarded  to  small 
business  by  program  participants  declined  since  fiscal  year  1996, 
more  recent  data  through  fiscal  year  2013  show  that  the  performance 
of  program  participants  has  remained  relatively  stable,  as  shown  in 
Figure  3.  Although  the  percentages  for  program  participants  are  lower 
than  for  nonparticipants,  the  trends  are  generally  the  same  for  both. 
The  differences  between  the  groups  may  be  attributable  to  some  of 
the  factors  we  discussed  in  our  report — such  as  the  large,  technically 
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complex  nature  of  some  participant’s  contracts  and  their  inability  to 
count  certain  small  business  subcontracting  activities  towards  their 
goals — which  do  not  suggest  shortcomings  in  the  program. 

•  Enhancing  opportunities  for  small  business  is  measured  by  more  than 
just  the  percentage  of  subcontract  dollars  awarded.  For  example,  the 
value  of  the  small  business  subcontracts  awarded  by  program 
participants  grew  during  this  period  from  about  $895  million  to  more 
than  $7.7  billion.  Small  businesses  therefore  have  received  a 
relatively  smaller  portion  of  a  significantly  larger  amount. 

•  As  discussed  in  the  report,  the  Test  Program  also  provided  enhanced 
opportunities  to  small  businesses  through  a  variety  of  initiatives 
completed  by  the  participants.  These  initiatives,  among  other  things, 
resulted  in  the  redirection  of  millions  of  dollars  of  subcontracts  from 
large  businesses  to  small  businesses,  and  increased  small  business 
participation  in  targeted  industries,  including  innovative  research. 

•  Finally,  regardless  of  the  overall  percentages,  we  found  that  program 
participants  met  or  nearly  met  all  of  the  “stretch”  percentage  goals 
negotiated  with  the  department  in  the  fiscal  years  we  reviewed.  In 
total,  the  Test  Program  participants  exceeded  the  dollar  goals  by 
approximately  $5.4  billion. 

Thus,  we  continue  to  believe  the  program  achieved  its  goals  of  enhancing 
opportunities  for  small  businesses,  and  as  discussed  in  the  report, 
reducing  participants’  and  DOD’s  administrative  costs.  Given  these 
findings,  and  that  DOD  had  not  acted  on  the  2007  review 
recommendation  to  work  with  Congress  to  make  the  program  permanent, 
we  included  a  recommendation  in  the  draft  report  for  DOD  to  draft  a 
legislative  proposal  to  make  the  program  permanent  or  otherwise  work 
with  Congress  to  determine  the  status  of  the  Test  Program.  The 
department  agreed  to  work  with  Congress  to  determine  the  status  of  the 
Test  Program;  however,  DOD  did  not  concur  with  the  recommendation  to 
draft  a  legislative  proposal.  Consequently  we  modified  the 
recommendation  and  added  a  matter  for  Congressional  consideration  to 
help  ensure  continued  reductions  in  administrative  costs  to  DOD  and 
program  participants  and  enhance  subcontracting  opportunities  for  small 
businesses. 
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We  are  sending  copies  of  this  report  to  the  appropriate  congressional 
committees  and  to  the  Secretary  of  Defense.  This  report  will  also  be 
available  at  no  charge  on  the  GAO  website  at  http://www.gao.gov. 

If  you  or  your  staff  have  any  questions  concerning  this  report,  please 
contact  me  at  (202)  512-4841  or  by  email  at  woodsw@gao.gov.  Contact 
points  for  our  Office  of  Congressional  Relations  and  Public  Affairs  may  be 
found  on  the  last  page  of  this  report.  Staff  who  made  key  contributions  to 
this  report  are  listed  in  appendix  III. 


William  T.  Woods 
Director 

Acquisition  and  Sourcing  Management 
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List  of  Committees 


The  Honorable  John  McCain 
Chairman 

The  Honorable  Jack  Reed 
Ranking  Member 
Committee  on  Armed  Services 
United  States  Senate 

The  Honorable  David  Vitter 
Chairman 

The  Honorable  Jeanne  Shaheen 
Ranking  Member 

Committee  on  Small  Business  and  Entrepreneurship 
United  States  Senate 

The  Honorable  Mac  Thornberry 
Chairman 

The  Honorable  Adam  Smith 
Ranking  Member 
Committee  on  Armed  Services 
House  of  Representatives 

The  Honorable  Steve  Chabot 
Chairman 

The  Honorable  Nydia  Velazquez 
Ranking  Member 
Committee  on  Small  Business 
House  of  Representatives 
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Appendix  I:  Objectives,  Scope,  and 
Methodology 


Section  821(e)  of  the  National  Defense  Authorization  Act  for  Fiscal  Year 
2015  included  a  provision  for  us  to  report  on  the  results  of  the  Department 
of  Defense’s  Test  Program  for  Negotiation  of  Comprehensive  Small 
Business  Subcontracting  Plans  (Test  Program).1  This  report  addresses 
the  extent  to  which  the  Test  Program  (1)  reduces  administrative  costs, 
and  (2)  enhances  subcontracting  opportunities  for  small  businesses. 

To  address  if  the  Test  Program  reduces  administrative  costs  for  program 
participants,  we  collected  and  analyzed  three  reviews  conducted  for  the 
Department  of  Defense  (DOD)  on  the  Test  Program  in  2002,  2007,  and 
2013  that  included  estimates  of  the  costs  associated  with  the  program. 

We  assessed  the  methodologies  and  assumptions  utilized — including  the 
number  of  contracts  and  cost  of  labor  to  compile  program 
documentation — and  used  data  collected  for  DOD’s  2013  study  to 
validate  the  findings.  To  help  assess  the  validity  of  the  data,  we  discussed 
with  DOD  officials  how  the  data  was  collected.  We  determined  that  the 
methodologies  were  valid  and  the  data  were  reliable  for  our  purposes.  In 
addition,  we  interviewed  DOD  officials  and  5  of  the  12  Test  Program 
participants  to  obtain  their  views  on  any  cost  savings  generated  by  having 
the  Test  Program,  as  well  as  any  benefits  they  experienced  from  a 
reduced  administrative  burden.  Test  Program  participants  we  interviewed 
represented  different  levels  of  participation  in  the  program,  including  two 
corporate-level  participants  and  three  division-level  participants,  as  well 
as  range  of  subcontracting  activity.  The  views  expressed  by  these 
participants  provided  insight  into  the  operation  of  the  program  but  are  not 
generalizable  to  all  program  participants. 

In  order  to  determine  if  the  conversion  costs  articulated  by  the  2013 
review  were  reasonable,  we  assessed  the  methodology  used,  obtained 
the  data  used  for  the  estimate,  discussed  how  the  data  was  collected  with 
DOD,  and  performed  our  own  analysis  using  more  recent  data  that  found 
the  conclusions  of  the  2013  review  to  be  reasonable.  To  perform  this 
analysis,  we  obtained  March  2015  contract  data  from  the  Defense 
Contract  Management  Agency  (DCMA)  for  the  Test  Program  participants. 
From  the  contract  listings  we  received,  we  removed  those  contracts  for 
which  we  could  not  determine  contract  costs,  as  well  as  those  that  were 
below  reporting  thresholds  that  would  require  an  individual  subcontracting 


^he  Carl  Levin  and  Howard  P.  “Buck”  McKeon  National  Defense  Authorization  Act  for 
Fiscal  Year  201 5,  Pub.  L.  No.  1 1 3-291 ,  §  821  (e). 
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Appendix  I:  Objectives,  Scope,  and 
Methodology 


plan,  to  determine  the  total  number  of  contracts  that  would  have  required 
individual  subcontracting  plans  in  the  absence  of  the  Test  Program.  DOD 
officials  also  provided  a  range  of  estimates  for  the  number  of  hours 
required  to  develop  an  individual  subcontracting  plan,  as  well  as  the  cost 
per  hour,  which  we  used  to  determine  the  cost  for  conversion.  To  validate 
the  March  2015  contract  data  received  from  DCMA,  we  also  obtained 
data  from  one  corporate-level  participant  with  a  high  level  of 
subcontracting  activity  that  we  interviewed.  This  allowed  us  to  estimate  its 
cost  for  converting  contracts  under  their  comprehensive  subcontracting 
plan  to  individual  subcontracting  plans.  We  determined  that  the  data  were 
reliable  for  our  purposes. 

To  address  whether  the  effect  of  Test  Program  initiatives  enhanced  small 
business  subcontracting  opportunities,  we  selected  the  period  of  fiscal 
years  2006  through  2013  for  analysis.  This  selection  was  made  because 
prior  DOD  reviews  released  in  2002  and  2007  assessed  participants’ 
performance  against  goals  and  initiatives  through  fiscal  year  2005  and 
fiscal  year  2013  was  the  last  full  year  of  data  available.  The  2002  study 
used  data  from  fiscal  years  1991  to  2000,  and  the  2007  study  used  data 
from  fiscal  years  2001  to  2005.  We  reviewed  the  available  plans, 
memorandums,  and  reviews  used  by  DCMA  to  assess  participant 
performance  from  fiscal  years  2006  through  2013.  This  included:  80 
annual  comprehensive  subcontracting  plans  submitted  by  participants  to 
DCMA,  60  DCMA  memorandums  documenting  negotiations  with  the 
participants,  and  85  annual  Form  640  reviews  performance  reviews  by 
DCMA.  We  also  analyzed  the  three  DOD-commissioned  reviews.  To 
provide  context  for  the  negotiation  process,  as  well  as  to  gather  different 
viewpoints  about  the  initiatives  as  a  whole,  we  interviewed 
representatives  from  DCMA’s  Small  Business  Programs  Division,  DOD’s 
Office  of  Small  Business  Programs  (OSBP),  and  five  Test  Program 
participants. 

To  address  whether  the  Test  Program  enhances  subcontracting 
opportunities  for  small  businesses  by  successfully  achieving  its  annual 
goals,  we  reviewed  Test  Program  participants’  comprehensive 
subcontracting  plans  and  negotiation  support  memorandums  and 
analyzed  performance  data  from  fiscal  years  2006  through  2013.  We 
compared  the  actual  performance  contained  in  the  Form  640  reviews 
against  the  approved  subcontracting  goals.  We  also  obtained  information 
about  the  small  business  subcontracting  performance  of  DOD  in  general 
from  DOD’s  OSBP.  Combining  this  information  with  that  obtained  for 
program  participants,  we  identified  trends  in  small  business 
subcontracting  for  Test  Program  participants  in  comparison  to  DOD  small 


Page  26 


GAO-16-27  Small  Business  Subcontracting 


Appendix  I:  Objectives,  Scope,  and 
Methodology 


business  subcontracting  performance  in  general.  We  also  reviewed 
legislation,  agency  guidance,  Federal  Register  notices,  and  relevant  GAO 
and  DOD  reports.  We  also  interviewed  officials  from  DOD’s  OSBP, 
DCMA’s  Small  Business  Programs  Division,  5  of  the  12  prime  contractors 
participating  in  the  Test  Program,  and  two  small  business  advocacy 
groups  chosen  for  their  representation  of  the  small  business  community, 
for  their  views  on  the  Test  Program,  and  factors  that  contribute  to  or 
undermine  its  success.  The  views  expressed  by  these  groups  provided 
insight  into  the  operation  of  the  program  but  are  not  generalizable  to  all 
small  businesses. 

We  conducted  this  performance  audit  from  March  to  November  2015  in 
accordance  with  generally  accepted  government  auditing  standards. 
Those  standards  require  that  we  plan  and  perform  the  audit  to  obtain 
sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for  our 
findings  and  conclusions  based  on  our  audit  objectives.  We  believe  that 
the  evidence  obtained  provides  a  reasonable  basis  for  our  findings  and 
conclusions  based  on  our  audit  objectives. 
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OFFICE  OF  THE  UNDER  SECRETARY  OF  DEFENSE 
3000  DEFENSE  PENTAGON 
WASHINGTON,  DC  20301-3000 


ACQUISITION. 
TECHNOLOGY 
AND  LOGISTICS 


OCT  3  0  m 


Mr.  William  T.  Woods 
Director 

Acquisition  and  Sourcing  Management 
U.S.  Government  Accountability  Office 
441  G  Street.  N.W. 

Washington.  DC  20548 

Dear  Mr.  Woods 

This  is  the  Department  of  Defense  (DoD)  response  to  the  Government  Accountability  Office 
(GAO)  Draft  Report.  GAO-16-27.  "SMAI.I.  BUSINESS:  DOD  Needs  to  Work  with  Congress  on 
Whether  the  Subcontracting  Plan  Test  Program  Should  Be  Made  Permanent.”  dated  September  30.  2015 
(GAO  Code  121272).  Detailed  comments  on  the  report  recommendations  are  enclosed. 


Sincerely, 


Kenyata  t^Wesley,  ActingfOirector 
Office  of  Small  Business  Programs 
Department  of  Defense 


Enclosure: 
As  stated 
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GAO  Draft  Report  Dated  September  30,  2015 
GAO-16-27  (GAO  Code  121272) 

“SMALL  BUSINESS:  DOI)  NEEDS  TO  WORK  WITH  CONGRESS  ON  WHETHER 
THE  SUBCONTRACTING  PLAN  TEST  PROGRAM  SHOULD  BE  MADE 
PERMANENT” 

DEPARTMENT  OF  DEFENSE  COMMENTS 
TO  THE  GAO  RECOMMENDATION 


RECOMMENDATION  I:  To  help  ensure  continued  reductions  in  administrative  costs  to 
Department  of  Defense  (DoD)  and  program  participants  and  enhance  subcontracting 
opportunities  for  small  businesses,  the  Government  Accountability  Office  (GAO)  recommends 
that  the  Secretary  of  Defense  draft  a  legislative  proposal  to  make  the  program  permanent,  or 
otherwise  work  with  Congress  to  determine  the  status  of  the  Test  Program.  In  doing  so.  the 
Secretary  could  provide  Congress  with  information  on  the  effectiveness  of  the  test  program  as 
discussed  in  the  three  DoD  commissioned  review  s  and  the  results  of  our  analysis. 

DoD  RESPONSE:  DoD  non-concurs  with  the  recommendation  to.  “draft  a  legislative  proposal 
to  make  the  program  permanent.”  and.  concurs  with  the  recommendation  to.  "work  with 
Congress  to  determine  the  status  of  the  Test  Program.”  DoD  supports  this  position  with  the 
following  response. 

Chart  1  below  shows  a  continuous  reduction,  over  two  decades,  in  the  percentage  of 
subcontracting  opportunities  for  small  businesses  from  all  Comprehensive  Subcontracting 
Program  (CSP)  participants.  This  analysis,  submitted  to  GAO  toward  the  end  of  the  study 
period,  contradicts  the  finding  that,  "the  Test  Program  enhanced  small  business  subcontracting 
opportunities.”  The  chart  does,  however,  support  the  GAO  recommendation  that  DoD  should, 
“work  with  Congress  to  determine  the  status  of  the  Test  Program.”  The  DoD  looks  forward  to 
working  with  the  Administration  and  Congress  with  respect  to  the  CSP.  We  believe  this  is  the 
best  approach  to  CSP  in  order  to  enhance  opportunities  for  small  businesses  of  the  defense 
industrial  base  and  reduce  administrative  costs  for  both  program  participants  and  DoD. 


1 
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Chart  1 :  The  data  collected  and  highlighted  for  three  key  years  shows  a  reduction  in  the 
percentage  of  small  business  subcontracting  dollars  awarded  to  small  business  concerns  by  the 
CSP  participants.  The  chart  shows  that  the  percentage  of  dollars  subcontracted  to  small 
businesses  by  all  of  the  CSP  participants  combined  declined  over  the  life  of  the  CSP  program 
from  approximately  46%  in  FY96  to  30%  in  FY07  to  24%  in  FY14.  This  trend  shows  that  the 
current  practices  associated  with  CSP  do  not  enhance  opportunities  for  small  businesses. 
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GAO  Contact 


William  T.  Woods,  (202)  512-4841  orwoodsw@gao.gov 


Staff 
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In  addition  to  the  contact  named  above,  J.  Kristopher  Keener  (Assistant 
Director),  Kathryn  (Emily)  Bond,  Joe  Hunter  (Analyst-in-Charge),  Cale 
Jones,  Julia  Kennon,  Stephen  V.  Marchesani,  Sylvia  Schatz,  and 
Roxanna  Sun  made  key  contributions  to  this  report. 
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GAO’s  Mission 

The  Government  Accountability  Office,  the  audit,  evaluation,  and 
investigative  arm  of  Congress,  exists  to  support  Congress  in  meeting  its 
constitutional  responsibilities  and  to  help  improve  the  performance  and 
accountability  of  the  federal  government  for  the  American  people.  GAO 
examines  the  use  of  public  funds;  evaluates  federal  programs  and 
policies;  and  provides  analyses,  recommendations,  and  other  assistance 
to  help  Congress  make  informed  oversight,  policy,  and  funding  decisions. 
GAO’s  commitment  to  good  government  is  reflected  in  its  core  values  of 
accountability,  integrity,  and  reliability. 

Obtaining  Copies  of 
GAO  Reports  and 
Testimony 

The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  at  no 
cost  is  through  GAO’s  website  (http://www.gao.gov).  Each  weekday 
afternoon,  GAO  posts  on  its  website  newly  released  reports,  testimony, 
and  correspondence.  To  have  GAO  e-mail  you  a  list  of  newly  posted 
products,  go  to  http://www.gao.gov  and  select  “E-mail  Updates.” 

Order  by  Phone 

The  price  of  each  GAO  publication  reflects  GAO’s  actual  cost  of 
production  and  distribution  and  depends  on  the  number  of  pages  in  the 
publication  and  whether  the  publication  is  printed  in  color  or  black  and 
white.  Pricing  and  ordering  information  is  posted  on  GAO’s  website, 
http://www.gao.gov/ordering.htm. 

Place  orders  by  calling  (202)  512-6000,  toll  free  (866)  801-7077,  or 

TDD  (202)  512-2537. 

Orders  may  be  paid  for  using  American  Express,  Discover  Card, 
MasterCard,  Visa,  check,  or  money  order.  Call  for  additional  information. 

Connect  with  GAO 

Connect  with  GAO  on  Facebook,  Flickr,  Twitter,  and  YouTube. 

Subscribe  to  our  RSS  Feeds  or  E-mail  Updates. 

Listen  to  our  Podcasts  and  read  The  Watchblog. 

Visit  GAO  on  the  web  at  www.gao.gov. 

To  Report  Fraud, 
Waste,  and  Abuse  in 
Federal  Programs 

Contact: 

Website:  http://www.gao.gov/fraudnet/fraudnet.htm 

E-mail:  fraudnet@gao.gov 

Automated  answering  system:  (800)  424-5454  or  (202)  512-7470 

Congressional 

Relations 

Katherine  Siggerud,  Managing  Director,  siggerudk@gao.gov,  (202)  512- 
4400,  U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room 

71 25,  Washington,  DC  20548 

Public  Affairs 

Chuck  Young,  Managing  Director,  youngd@gao.gov,  (202)512-4800 

U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7149 
Washington,  DC  20548 
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